Evaluating the Forecasts of Risk Models

The forecast evaluation literature has traditionally focused on methods for assessing point-forecasts. However, in the context of risk models, interest centers on more than just a single point of the forecast distribution. For example, value-at-risk (VaR) models which are currently in extremely wide use form interval forecasts. Many other important financial calculations also involve estimates not summarized by a point-forecast. Although some techniques are currently available for assessing interval and density forecasts, none are suitable for sample sizes typically available. This paper suggests an new approach to evaluating such forecasts. It requires evaluation of the entire forecast distribution, rather than a value-at-risk quantity. The information content of forecast distributions combined with ex post loss realizations is enough to construct a powerful test even with sample sizes as small as 100. Acknowledgements: I gratefully acknowledge helpful input from Peter Christoffersen, Michael Gordy, Matt Pritsker and Jim O’Brien. Any remaining errors and inaccuracies are mine. The opinions expressed do not necessarily represent those of the Federal Reserve Board or its staff. See, Jorion (1997) for a recent survey and Risk Publications’ VAR: Understanding and 1 Applying Value-at-Risk for a compendium of research papers. In addition to the CME, the Chicago Board of Trade, the London Futures and Options 2 exchange and at least a dozen other exchanges currently use SPAN. See Kupiec (1993) for a detailed description. 2